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EXAMINER'S ANSWER 



This is in response to the appeal brief filed September 14, 2007 appealing from the Office action 
mailed February 22, 2007. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings 
which will direcdy affect or be directiy affected by or have a bearing on the Board's decision in the 
pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

* (4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection contained in the 
brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 



(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
substantially correct. The changes are as follows: : 
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WITHDRAWN REJECTIONS 

The following grounds of rejection are not presented for review on appeal because they have 
been withdrawn by the examiner. 

• Rejections of claims 33-48 under 35 USC § 112, first paragraph. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

5,761,339 Ikeshoji et al 6-1998 

5,778,092 Macleodetal 7-1998 

5,065,437 Bloomberg 11-1991 

Applicant's admitted prior art (here AAA) - background of instant application no. 10/064,873. 

Gonzales et al, 1992, Book Publication "Digital Image processing". 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 



Claim Refections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

2. Claim 33 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al., U.S. 
Patent No. 5,761,339, further in view of applicant's admitted prior art (here AAA). 

Regarding claims 33, Ikeshoji discloses acquiring an a grayscale image data in 
which textual annotations are present, using a scanner and then storing such data on a recording 
medium or further displaying it on a display and further discloses that the present invention relates 
to an image processing method which is suitable for correction of photograph or a picture or a 
document of characters (col. 1, lines 7-13; col. 2, lines 43-50). Now as per the claims, the claim 
limitations "acquiring data representing a grayscale image; adding data representing a textual 
annotation to said acquired grayscale image data" discloses acquiring the grayscale image first and 
then adding the annotations to said acquired grayscale image, which results in an annotated grayscale 
image. Ikeshoji's invention as discussed in the previous rejections as well as described here in this 
rejection is directed to enhancing or correcting images that are annotated grayscale images. Ikeshoji 
as cited above acquires an annotated grayscale image (which is manually or non-electronically 
annotated) but does not expressly teach the steps of electronically adding the annotations to the 
electronic grayscale image to obtain annotated grayscale image. However, it is very well known to 
generate annotated images and applicant has admitted in the background of the invention that such 
a process of generating an annotated image is very well known and is used in many application such 
as medical diagnostic imaging (AAA teaches, "In many applications, such as medical diagnostic 
imaging, images are saved with annotations burnt in. The annotations are typically burnt in by 
overlaying an arbitrary intensity value of text on the image" (para. 0002)). As well known in the 
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medical diagnostic imaging patient's name along with other information is added to the image itself 
for the purpose of image labeling for the identification purposes. Further for the sake of arguments, 
examiner directs applicant to softwares like Adobe Photoshop and Microsoft paint which are very 
well-known to be used for image editing, including annotation. Ikeshoji teaches generating a 
annotated image where the annotations are added to the paper and then the paper is scanned to 
generate an annotated image and AAA on the other hand, teaches scanning the image and then 
electronically adding the annotations to obtain an annotated image. Since both ways of obtaining an 
annotated image are very well known to be as prior art and applicant itself has agreed to such 
methods, therefore, it would have been obvious for one of ordinary skill in the art at the time of 
invention was made to use any of the method defined by the prior arts and it would be obvious to 
one of ordinary skill in the art at the time of invention was made to use AAA's teachings in the 
invention of Ikeshoji because both references teach generating annotated images and AAA's 
teachings would prevent the original image to be permanendy annotated and AAA presents the ease 
of annotating the images since no paper and pen is required, however in the case of Ikeshoji the 
original paper annotated image will retain the annotations permanently and everything is done 
manually. 

Ikeshoji discloses removing data representing said textual annotation from said stored data 
representing said annotated grayscale image to derive data representing an unannotated grayscale 
image (figure 1; element 20D being the first modified image, which is obtained after annotations 
(elements 30, 32 and 34) are removed; element 30D represents removed annotations where 
annotations being textual (characters) (See Title; figure 1 clearly says image 30D being character and 
figure image; col. 2, lines 36-42 discloses characters (textual annotations) are drawn on an image). 
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Ikeshoji further discloses processing said data representing an unannotated grayscale image 
(20D) using an algorithm to derive data representing a processed grayscale image (col 5, lines 1-5; 
removing stains or making the stain inconspicuous). 

Ikeshoji further discloses merging said removed data representing said textual annotation 
and said data representing said processed grayscale image, said merged data representing an 
annotated processed grayscale image (figure 3, element 10-1 being the merged image). 

3. Claims 34-44 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al, 
U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and further in view of 
Macleod et al, U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. Patent No. 
5,065,437. 

Claim 34 recites "The method as recited in claim 33, wherein said removing step comprises 
the following: deriving data representing a first binary mask defining one or more image regions; and 
multiplying said data representing said first binary mask and said data representing said annotated 
grayscale image to derive said unannotated grayscale image". Ikeshoji clearly teaches in figure 1, 
deriving a first modified image (element 20 D) from which annotations have been removed with the 
help of filtering (or masking). Removing a part of images by using a mask is very well known in the 
process of image segmentation where processing such as convolution further involving multiplying 
mask bits with original image is used and is generally used in differentiating and removing an image 
part from the original image. Ikeshoji does teach of using a filter or mask but does not specifically 
teach deriving a binary mask, however, Macleod teaches this generally used well-known method of 
deriving a binary mask (col. 2, lines 1-13; col. 4, lines 65-68 through col. 5, lines 1-16 and lines 30-68; 
col. 8, lines 4-10 and lines 25-40). 
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Further supporting, in lines 57-62 of col. 11, MacLeod discloses that U.S. Patent No. 5,065,437, is 
incorporated by reference, thereby, not requiring the motivation to combine the references. 
Bloomberg. Bloomberg provides the support for subject matter in claim 2 in (figure lB,col. 2, lines 
20-68; col. 4, lines 1-68; col. 6, lines 10-40). Therefore, it would have been obvious for one of 
ordinary skill in the art at the time of invention was made to use combined teachings of MacLeod 
and Bloomberg in the invention combined of Ikeshoji and AAA because all references are directed 
to image segmentation and the combined MacLeod and Bloomberg's teachings provide the well- 
known teachings that are generally used in image segmentation, which would further provide a 
robust and computationally efficient technique for identifying and separating regions (See 
Bloomberg, col. 2, lines 5-15). 

Claim 35 recites "The method as recited in claim 34, wherein said merging step comprises 
the following: inverting said data representing said first binary mask to derive data representing a 
second binary mask defining one or more annotation regions; multiplying said data representing said 
second binary mask and said data representing said annotated grayscale image to derive data 
representing a modified image; and merging said data representing said modified image and said data 
representing said processed grayscale image to derive said data representing said annotated grayscale 
image". Ikeshoji clearly shows in figure 1, deriving images 20D and 30D, where in image 20D 
annotations are removed and in image 30D only annotations are present, therefore these 2 images 
are clearly subject to the apparent use of a mask that when inverted would provide segmentation for 
extracting one part or the other from the original image. Figure 3 clearly shows merging said second 
modified image (30) and said processed image (20) to derive said merged image. 
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Claim 36 recites "The method as recited in claim 33, wherein the merged textual 
annotations occupy the same pixels in said annotated processed grayscale image that the removed 
textual annotations originally occupied in said annotated grayscale image". Ikeshoji clearly shows in 
figure 3 the image 10-1 which has annotation at the same image position as shown in figure 1 image 
10 and further support can be found in (col. 4, lines 6-12). 

Claim 37 recites "the method as recited in claim 33, wherein said removing step comprises 
morphology-based processing and thresholding". MacLeod discloses dilation and erosion 
(morphological operations) (col. 7) and further teaches thresholding (col. 5). 

Claim 38 recites "The method as recited in claim 33, wherein said removing step comprising 
the following: grayscale erosion of said data representing said annotated grayscale image using a 
structuring element to derive data representing an eroded grayscale image; thresholding said data 
representing said eroded grayscale image to derive data representing a first binary mask; dilation of 
said data representing said first binary mask using said structuring element to derive data 
representing a second binary mask defining one or more image regions; and multiplying said data 
representing said second binary mask and said data representing said annotated grayscale image to 
derive said data representing said image". The subject matter recited in the claim 38 is nothing but a 
morphological operation (opening: erosion followed by dilation), which is very well-known to be 
used in the process of identifying and separating image portions. MacLeod clearly teaches all the 
steps recited in claim 6 in (col. 11 through col. 12, lines 1-55). Bloomberg further provides the 
support in (col. 9, lines 45-68 through col. 10, lines 1-25). 
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Regarding claim 39, citing the well-known fact of modifying the image using an image 
mask, and in further view of Ikeshoji providing merging of images as explained before in the 
rejection of claim 3. Claim 7 has been similarly analyzed and rejected as per claims 35 and 38. 

Regarding claim 40, MacLeod discloses said removing step comprises thresholding and 
pixel connectivity-based analysis" (col. 6, lines 44-56). See Bloomberg (col. 2, lines 28-60; col. 4, lines 
40-65; col. 7, lines 1-65). 

Regarding claim 41, claim 41 additionally recites the limitation where "using 8-connected 
analysis to reject segments smaller than a prespecified size from said first binary mask to 
derive a second binary mask". MacLeod discloses of the same in (col 6, lines 44-56; col. 10, lines 1- 
40). All other limitations have been similarly analyzed and rejected as per claims 33-40. 

Regarding claim 42, claim 42 has been similarly analyzed and rejected as'per claims 41, 39 

and 35. 

Regarding claim 43, claim 43 in addition to claim 41 recites "removing holes from a 
second binary mask to derive a third binary mask". MacLeod teaches mask reconstruction for filling 
(removing) the interior holes (col. 11, lines 63-68 through col. 12, lines 1-15). 

Regarding claim 44, Ikeshoji discloses filtering to enhance said modified ultrasound image 
(col. 6, lines 10-20). 
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4. Claims 45-48 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al., 
U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and further in view of 
Macleod et al, U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. Patent No. 
5,065,437 and further in view of Gonzales et al, 1992, Book publication "Digital image processing". 

Regarding claim 45, as discussed before in the rejection of claims 33-44, the combined 
invention of Ikeshoji, AAA, MacLeod and Bloomberg provides techniques that are used on 
grayscale images. Now the question is, can the same techniques be used on color images? The 
answer would be yes and further support is provided by Gonzales. Gonzales teaches "the HIS 
model is ideally suited for image enhancement, because the intensity component is decoupled from 
the color information in an image. Therefore any monochrome enhancement technique can be 
carried over as a tool for enhancing full-color images. It simply calls for converting the image to the 
HIS format, processing the intensity component, and converting the result to RGB for display" 
(page 247, 4 th paragraph) and apparently the HS components would have to be merged back in as 
the processing is done on color images and the output required would be a color image. Also, 
converting an RGB to HSI and HSI to RGB model is very well known and is further taught by 
Gonzales on pages 229 and 235. Therefore, providing a motivation for one of ordinary skill in the 
art at the time of invention was made to combine Gonzales's teachings in the combined invention 
of Ikeshoji, AAA, MacLeod and Bloomberg. All other limitations of claim 45 have been similarly 
analyzed and rejected as per claims 33-44. 

Claims 46-48 have been similarly analyzed and rejected as per claims 34-45. 
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(10) Response to Argument 

Summary of Arguments: 

1 . Regarding independent claim 33, 
(i) Appellant argues in substance on pages 7-8 of the appeal brief filed: 

Ikeshoji neither discloses nor suggests that annotations are added to the image 
acquired by the scanner before the scanner-acquired image undergoes electronic 
processing . 

Examiner's Response : 

In regards to item (i), Examiner contends the following: 
Claim 33 recites: 

(a) generating an annotated image by adding textual annotation to an grayscale image (or any 

image), 

(b) displaying and storing the annotated image, 

(c) removing the annotation from the image to get un-annotated image, 

(f) processing the un-annotated image using some algorithm, and then 

(g) merging back the removed annotation with the processed un-annotated image to 
generate an annotated processed image. 



Page 1 1 
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Examiner rejected claim 33 over Ikeshoji further in view of Applicant's admitted prior art 
(here AAA). As from the rejections made, Ikeshoji teaches all the steps from (b) through (g) as 
shown above. Ikeshoji is combined with AAA only for step (a). Examiner cited (col. 1, lines 7-13; 
col. 2, lines 43-50) in Ikeshoji which clearly teaches acquiring an annotated grayscale image (such as 
document of characters and/or figures) using a scanner and then storing such data on a recording 
medium or further displaying it on a display and further discloses that the present invention relates 
to an image processing method which is suitable for correction of photograph or a picture or a 
document of characters (col 1, lines 7-13; col. 2, lines 43-50). Examiner further asserted that 
Ikeshoji as cited acquires an annotated grayscale image (which is manually or non-electronically 
annotated) but does not expressly teach the steps of electronically (though the claims do not 
call for electronically adding the annotation) adding the annotations to the electronic 
grayscale image to obtain annotated grayscale image. However, it is very well known to 
generate annotated images and applicant has admitted in the background of the invention that such 
a process of generating an annotated image is very well known and is used in many application such 
as medical diagnostic imaging (AAA teaches, "In many applications, such as medical diagnostic 
imaging, images are saved with annotations burnt in. The annotations are typically burnt in by 
overlaying an arbitrary intensity value of text on the image" (para. 0002)). Ikeshoji teaches generating 
an annotated image where the annotations are added to the paper and then the paper is scanned to 
generate an annotated image and AAA on the other hand, teaches scanning the image and then 
electronically adding the annotations to obtain an annotated image. Since both ways of obtaining an 
annotated image are very well known to be as prior art and applicant itself has agreed to such 
methods, therefore, it would have been obvious for one of ordinary skill in the art at the time of 
invention was made to use any of the method defined by the prior arts and it would be obvious to 
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one of ordinary skill in the art at the time of invention was made to use AAA's teachings in the 
invention of Ikeshoji because both references teach generating annotated images and AAA's 
teachings would prevent the original image to be permanendy annotated and AAA presents the ease 
of annotating the images since no paper and pen is required, however in the case of Ikeshoji the 
original paper annotated image will retain the annotations permanendy and everything is done 
manually. 

In item (i), Appellant's arguments are simply directed to the reference Ikeshoji on the same 
subject matter of which examiner has agreed (such as - Ikeshoji as cited acquires an annotated 
grayscale image (which is manually or non-electronically annotated) but does not expressly teach 
the steps of electronically adding the annotations to the electronic grayscale image to obtain 
annotated grayscale image), and no comments /arguments on the combination of Ikeshoji and 
AAA have been, found in the appellant's arguments, therefore persuasive arguments were not found 
in the appeal brief Examiner does not rely on Ikeshoji for providing the teachings of adding the 
annotations to the image but relies on the AAA for such teachings. 

(ii) Appellant further argues in substance: 

Nor does Ikeshoji disclose or suggest adjusting the brightness or contrast of the 
scanner-acquired image. Appellants submit that it would not have been obvious to apply 
Ikeshoji's technique to situations where one desires to improve the contrast or brightness of 
an acquired image that had annotations added to it after image acquisition . 

Examiner's Response : 

In regards to item (ii), Examiner contends the following: 
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Claim 33 as claimed recites the limitation "processing said data representing said un- 
annotated grayscale image using an algorithm to derive data representing a processed grayscale 
image''. Clearly, the claim 33 limitation does not call for adjusting the brightness or contrast of the 
image but a broader processing of an image which can include any type of image processing. 
Therefore, appellant's arguments are not relevant with respect to the claimed subject matter as 
recited in claim 33. 

In the rejection made, Ikeshoji discloses processing said data representing an unannotated 
grayscale image (20D) using an algorithm to derive data representing a processed grayscale image 
(col. 5, lines 1-5; from background image removing stains or making the stain inconspicuous) and 
for the arguments sake further adding, stain removal process is nothing but adjusting the brightness 
or contrast of the image which changes the image properties or characteristics. 

(iii) Appellant further argues in substance: 

It would not be obvious to transfer any teachings of Ikeshoji. which deals with 
enhancing physical documents by converting them into electronic images, to the field of 
electronically annotated electronic images not derived by scanning a physical document . 

Examiner's Response: 

In regards to item (iii), Examiner contends the following: 

First of all, claim 33 does not call for any specific image type and therefore appellant's 
arguments are not relevant with respect to a specific image type. Further adding, the claim does not 
call for electronically adding the annotations to the image in first place. As discussed before in the 
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rejection, Ikeshoji same as instant application, is directed to removal of annotations from an 
electronic image and then processing the un-annotated electronic image. The only difference 
between Ikeshoji and the instant invention as claimed in claim 33 is that the instant invention adds 
annotations to the image electronically and Ikeshoji teaches an image that already recites annotations 
and examiner in the above response and rejections has already provided response why one would 
prefer a first method over a second method for adding annotations to the image. Ikeshoji does not 
enhance physical documents but the electronic images of the documents and since the processing 
being electronic, same image processing functions can be applied to any other electronic image, 
since all electronic images are nothing but collection of pixels. Further adding, claim 33 is a Jepson 
claim and Appellant has agreed on page 6 of appeal brief that adding the annotations is the prior art 
and as per the claims and specification originally filed the invention claimed is only directed to 
enhancing the annotated images. Therefore, appellant is arguing on the subject matter that instant 
invention does not possess. 

2. Regarding independent claim 45, 

(i) Appellant argues in substance on page 9 of the appeal brief filed: 

Ikeshoji teaches colored stains can be removed from the images but Ikeshoji neither 
discloses nor suggests removing the hue and saturation data from a color image, processing 
the remaining brightness component, and then restoring the hue and saturation 
components. Applicants respectfully submit it would not be obvious to combine the color 
processing techniques disclosed in the Gonzalez article with Ikeshoji. 



Examiner's Response : 



Application/Control Number: 10/064,873 Page 16 

Art Unit: 2624 

Examiner contends the following: 

Gonzalez presents a digital image-processing book very well known from last 20 years and 
has been widely used. Examiner cited a portion of Gonzales's book on (page 247, 4 th paragraph) 
which teaches "the HIS model is ideally suited for image enhancement, because the intensity 
component is decoupled from the color information in an image. Therefore any monochrome 
enhancement technique can be carried over as a tool for enhancing full-color images. It simply calls 
for converting the image to the HIS format, processing the intensity component, and converting the 
result to RGB for display". In other words, Gonzales teaches that any monochrome enhancement 
technique can be used for enhancing full color images. Gonzales further teaches processing the HIS 
(H-Hue component, I-Brightness (intensity) component, S-Saturation component) image in which 
HS components are separated from I component and then I component which being the brightness 
component is processed and therefore same enhancement techniques applied to the monochrome 
or grayscale images can also be applied to the color images. Therefore, the motivation for a person 
of ordinary skill in the art at the time of invention was made for combining Gonzales with Ikeshoji 
itself lies in the Gonzales reference. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related 
Appeals and Interferences section of this examiner's answer. 
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For the above reasons, it is believed that the rejections should be sustained. 




Dated: November 19, 2007 
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Supervisory Patent Examiner 
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